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Abstract 


This study seeks to understand how to use formal learning activities to effectively 
support the development of open education literacies among K-12 teachers. Considering 
pre- and post-surveys from K-12 teachers (n = 80) who participated in a three-day 
institute, this study considers whether participants entered institutes with false 
confidence or misconceptions related to open education, whether participant knowledge 
grew as a result of participation, whether takeaways matched expectations, whether 
time teaching (i.e., teacher veterancy) impacted participant data, and what specific 
evaluation items influenced participants’ overall evaluations of the institutes. Results 
indicated that 1) participants entered the institutes with misconceptions or false 
confidence in several areas (e.g., copyright, fair use), 2) the institute was effective for 
helping to improve participant knowledge in open education areas, 3) takeaways did not 
match expectations, 4) time teaching did not influence participant evaluations, 
expectations, or knowledge, and 5) three specific evaluation items significantly 
influenced overall evaluations of the institute: learning activities, instructor, and 
website / online resources. Researchers conclude that this type of approach is valuable 
for improving K-12 teacher open education literacies, that various misconceptions must 
be overcome to support large-scale development of open education literacies in K-12, 
and that open education advocates should recognize that all teachers, irrespective of 
time teaching, want to innovate, utilize open resources, and share in an open manner. 
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Introduction 


Despite decades of work in the area and hundreds of initiatives and research studies 
focused on utilizing technology to improve classroom teaching and learning, effective 
technology integration remains a “wicked problem,” complicated by diverse learning 
contexts, emerging technologies, and social trends that make formalized approaches to 
technology integration and theory development difficult (Kimmons, in press; Mishra & 
Koehler, 2007). Within this space, those intent upon improving K-12 teaching and 
learning with technology have had difficulty agreeing upon what constitutes effective 
integration, what the purposes of integration might be, and how such integration might 
help to solve some of the persistent problems plaguing educational institutions without 
falling prey to technocentric approaches to change (Papert, 1987). 

In response to technocentrism, open education has arisen as an approach for integrating 
technology into the learning process with a vision for “building a future in which 
research and education in every part of the world are ... more free to flourish” (Budapest 
Open Access Initiative, 2002, para. 8) and increasing “our capacity to be generous with 
one another” (Wiley, 2010, para. 39). That is, technology in open education is seen 
merely as a tool for encouraging and empowering openness. As such, open education 
encompasses a variety of movements and initiatives, including open textbooks (Baker, 
Thierstein, & Fletcher, 2009; Hilton & Laman, 2012; Petrides, Jimes, Middleton- 
Detzner, Walling, & Weiss, 2011) and other open educational resources (Atkins, Brown, 
& Hammond, 2007; OECD, 2007; Wiley, 2003), open scholarship (Garnett & 
Ecclesfield, 2012; Getz, 2005; Veletsianos & Kimmons, 2012), open access publishing 
(Furlough, 2010; Houghton & Sheehan, 2006; Laakso, 2011; Wiley & Green, 2012), and 
open courses (Fini, 2009; Kop & Fournier, 2010; UNESCO, 2002). 

Most proponents of open education focus exclusively upon higher education, despite 
much excitement among teachers for expanding open practices to K-12 and preliminary 
evidence that open education can help to address persistent K-12 problems. Reasons for 
lack of spill-over into K-12 vary, but it is likely that this difference stems in part from the 
fact that change in K-12 must either occur at the highly bureaucratic state level or at the 
hidden local level, whereas higher education institutions and their professors have more 
flexibility to try innovative approaches and also enjoy greater visibility for sharing 
results. Nonetheless, advances are being made in bringing open practices to K-12 
through both practice and research. 

Perhaps the most well-known study in this regard was completed by Wiley, Hilton, 
Ellington, and Hall (2012), wherein they conducted a preliminary cost impact analysis 
on K-12 school use of open science textbooks and found that these resources may be a 
cost-effective alternative for schools if certain conditions are met (e.g., high volume). 
Beyond driving down costs, however, others have suggested that open education can 
help support the emergence of “open participatory learning ecosystems” (Brown & 
Adler, 2008, p. 31), can counterbalance the deskilling of teachers that occurs through 
the purchasing of commercial curricula (Gur & Wiley, 2007), and can provide a good 
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basis for creating system-wide collaborations in teaching and learning (Carey & Haney, 
2007). These potentials represent promising aims for K-12 and have even led to the 
development of open high schools intent upon democratizing education and treating 
access to educational materials as a fundamental human right (Tonks, Weston, Wiley, & 
Barbour, 2013). 

However, it is also recognized that the shift to open is problematic for a number of 
reasons (Baraniuk, 2007; Walker, 2007), not least of which is the fact that K-12 teachers 
must develop new information literacies to become effective open educators (Tonks, 
Weston, Wiley, & Barbour, 2013), and little work has been done to study how to best 
support these professionals in developing literacies and practices necessary to embrace 
openness or to utilize and create their own open educational resources (cf. Jenkins, 
Clinton, Purushotma, Robinson, & Weigel, 2006; Rheingold, 2010; Veletsianos & 
Kimmons, 2012). If advocates of open education seek to diffuse open educational 
practices, then a lack of understanding in how to support literacy development among 
K-12 teachers is a clear problem. To combat this, this study seeks to move forward the 
state of the literature and practice on how to effectively train teachers in developing 
open education literacies. 

As personnel in a center for innovation and learning at a public university in the United 
States, the researchers have taken on the challenge of improving K-12 teaching and 
learning in their state through effective technology integration and believe that open 
education may be a way forward for enacting real, scalable change in public K-12 
schools. They also believe that open education can serve as an empowering vision that 
schools may use to move ahead with meaningful technology integration initiatives. 
However, open education is a new concept to most K-12 teachers and administrators, 
and knowledge and skills necessary for effectively utilizing and creating open 
educational resources are not standard topics of teacher education courses or 
professional development trainings. 

As a result, the researchers have sought to push forward a new, grassroots initiative in 
their state focused upon helping K-12 personnel to develop the knowledge, skills, and 
attitudes necessary for becoming effective open educators. The first wave of this 
initiative consisted of conducting a series of Technology and Open Education Summer 
Institutes for K-12 teachers in the target state, wherein over one hundred teachers 
participated in a 3-day collaborative learning experience focused on learning about 
issues related to open education (e.g., copyright, copyleft, Creative Commons) and 
creating and remixing their own open educational resources. 

As we conducted these institutes, we faced a number of challenges and uncertainties due 
to lack of previous work in this area. Some of these included wondering 1) whether 
participants entered the institutes with an accurate understanding of open education 
concepts, 2) whether such an institute setting could be effective for increasing teacher 
knowledge in this area, 3) whether participant takeaways would match their 
expectations, 4) whether time teaching or teacher veterancy had any impact on 
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participant perceptions of the learning experience, and 5) what factors might influence 
an overall evaluation of the institute as a valuable learning experience. 

In this study, we explored five research questions emerging from these concerns which 
will help to inform on-going efforts to promote open education practices in K-12. These 
questions included the following: 

RQi. Did participants enter the institute with false confidence or 
misconceptions related to open education concepts (e.g., copyright)? 
RQ2. Did participant self-assessments of open education knowledge grow as 
a result of the institute? 

RQ3. Did participant takeaways match initial expectations or change as a 
result of the institute? 

RQ4. Did time teaching (i.e., teacher veterancy) have an effect on 
participants’ expectations, knowledge, or evaluation metrics? 

RQ 5 . What specific evaluation items influenced participants’ overall 
evaluations of the institute? 

Background 

As part of our mission to improve K-12 teaching and learning with technology, our 
research team conducted a series of three-day Technology and Open Education Summer 
Institutes with K-12 teachers in our state. Each institute involved up to 30 participants 
and was organized according to grade level, with two institutes focusing on elementary 
and two focusing on secondary education. 

In total, over one hundred K-12 teachers from all over the target state participated in the 
summer institutes, representing all grade levels, a variety of subject areas, and all of the 
state’s educational regions (cf. Idaho State Department of Education, 2007). To our 
knowledge, there has never been any professional development experience quite like 
this attempted anywhere, in terms of subject, scale, scope, and diversity of participants, 
and this study extends prior work in this area by introducing and evaluating an 
approach to supporting K-12 teacher open education literacy development that is not 
bounded by a single school or subject area. The overarching institute vision was to help 
educators across the state to develop open education literacies that they could then take 
back to their schools for enacting change and supporting innovation in open educational 
practices. Such a grassroots, broad-spectrum approach to open education is unique and 
untested, and our goal was to yield research outcomes that could help us move forward 
with ongoing innovation in this area. 

When applying to attend the institutes, potential participants identified subject areas 
and grade levels that were of most interest to them. If accepted, participants were then 
assigned to a professional learning community or PLC (DuFour, 2004) within their 
institutes that was focused on their subject area and/or specific grade level. This meant 
that though each institute was either focused on elementary or secondary education, 
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each participant had a focused experience in one of five PLCs. PLC focus areas varied by 
institute but typically included subject area specialization (e.g., science, mathematics). 

The actual structure of learning activities at each institute was also atypical as compared 
to most K-12 professional development experiences. Each institute consisted of roughly 
3 phases or days. Day One was more traditional in the sense that it was largely 
instructor-centered and focused on presentations, provocative videos, and class-wide 
discussions. During Day One, a small portion of the time was also devoted to helping 
participants to get to know their PLCs and to begin making plans for how they would 
work together through the institute. Day Two was completely different. At the start, 
participants immediately took a few minutes for a planning session with their PLCs to 
set goals and to gather thoughts from the day before and then began a series of 
development sprints where each PLC worked together to create open educational 
resources that would be valuable to their members’ schools and classrooms. During Day 
Two, the instructor interjected occasionally to provide guidance and support, but all 
learning and activities were driven by the goals established by each PLC autonomously. 
During Day Three, the PLCs were given time to wrap up their projects, the instructor 
provided final guidance on sharing, and each PLC presented their products to the larger 
group and also made their resources available to the public on the web. 

Throughout this process, technology was heavily used to support collaboration and 
communication. The open course website was made available to participants and the 
public before the institute began and remains open and available indefinitely 
(Kimmons, 2014). This decision was surprising to participants, who were accustomed to 
professional development experiences where information was initially provided but 
severed upon completion. Making information and resources perpetually available to 
participants gave them more freedom to focus on working on their own products and 
critically evaluating learning experiences as opposed to spending time laboriously taking 
notes in preparation for the time when access to information resources would cease. 

Within the lab space utilized for the institutes, each PLC was assigned to a horseshoe¬ 
shaped table with a display switching matrix and large-screen interactive display along 
with personal computing devices to connect into their tables. This allowed each 
participant to wirelessly access information resources and work on institute materials 
individually but also to work within the context of a group setting where they could 
autonomously and effectively collaborate, share, and present their information to other 
group participants. Throughout this process, collaborative document creation software 
(i.e., Google Drive) was used so that participants could work on the same documents 
simultaneously and share resources in a common, cloud-based folder. 

Before these institutes, many participants had never experienced using these types of 
software and hardware tools before, and most had never used them in a synchronous, 
collaborative setting. Furthermore, the lab also provided access to a variety of other 
cutting-edge technologies like an interactive table, wearable devices, a telepresence 
videoconferencing robot, and kinesthetic learning games, which participants were given 
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opportunities to try out and consider their applications for local school use during 
Technology Exploration sessions. 

Though a variety of technologies were provided, technology was not the focus of the 
institutes but was rather a tool that was used to inspire participants to think creatively 
and to collaborate in open ways. Because it was anticipated that most teachers would 
have had little exposure to open education, technology was also used as a marketing tool 
for the institute, because though most teachers may not have had initial interest in the 
unknown topic of open education, it was expected that access to new technologies would 
be a motivator for eliciting interest in the institutes. 


Methods 


This study employed a longitudinal survey design methodology (Creswell, 2008) to 
collect and analyze data from institute participants before and after the institute. This 
method was deemed to be appropriate, because research questions lent themselves to 
quantitative analysis of trends among institute participants over the course of the three- 
day experience. 

Sample 

Survey respondents included eighty (n = 80) participants in the targeted Technology 
and Open Education summer institutes. In total, over one hundred K-12 educators 
participated in the institutes, but not all elected to participate in the study. Participants 
were predominantly female, reflecting an uneven gender distribution of the K-12 labor 
force in the target state, came from all geographic regions of the target state, and were 
generally veteran teachers (72% having taught for five or more years). More detailed 
participant demographic information was not collected, because it was deemed 
unnecessary to answer the research questions. 

Data Collection 

Throughout the institutes, both quantitative and qualitative feedback was elicited from 
participants, but this report deals primarily with quantitative results. Data sets for this 
study included two online surveys: one conducted immediately before the institute and 
one conducted immediately after the institute. 

Survey Instruments 

Both surveys were delivered online, and participants completed them by following a link 
on their personal or provided laptops or mobile devices while at the institute. Surveys 
consisted of a number of questions that may be categorized as eliciting one of the 
following: 
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• fact (e.g., years teaching); 

• expectation (e.g., personal learning goal); 

• knowledge (e.g., self-assessment); 

• evaluation (e.g., instructor evaluation); 

• open response (e.g., general feedback). 

Pre-survey. 

The pre-survey consisted of the following two factual questions, knowledge question, 
and expectation question: 

1. How long have you been teaching? [Fact] 

2. Did you do the preliminary work for this institute, including watching 
the videos, creating a Google account, and reading the articles? [Fact] 

3. How well do you understand each of the following concepts or 
movements (explained below)? [Knowledge] 

4. What do you hope to gain from this institute (explained below)? 
[Expectation] 

The knowledge question consisted of six separate items and yielded a reliable 
Cronbach’s alpha of .74. 

Post-survey. 

The post-survey consisted of two knowledge questions, five evaluation questions, one 
expectation question, and three open response questions: 

1. How well did you understand each of the following concepts or 
movements before the institute (explained below)? [Knowledge] 

2. How well do you understand each of the following concepts or 
movements now (explained below)? [Knowledge] 

3. Compared to other professional development sessions, this institute 
was: (Leave blank if this is your first professional development session.) 
[Evaluation] 

4. How would you rate this institute (explained below)? [Evaluation] 

5. The institute was a good use of my time. [Evaluation] 

6. The institute was of practical value to me in the classroom. [Evaluation] 

7. This institute helped me to think critically about how I incorporate 
technology into my teaching. [Evaluation] 

8. What was the most valuable knowledge or skills that you gained from 
this institute (explained below)? [Expectation] 

9. What was most valuable about this institute? [Open response] 

10. What would you change about this institute? [Open response] 

11. Please provide any feedback or suggestions to help us understand how 
to make future institutes meaningful and valuable for teachers. [Open 
response] 
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Knowledge questions each consisted of six separate evaluations and yielded a reliable 
Cronbach’s alpha of .86. Evaluation questions consisted of fifteen total items and 
yielded a reliable Cronbach’s alpha of .85. 

Response rate. 

A complete response was determined by the presence of both a pre-survey and post¬ 
survey for each participant. Since all study participants were encouraged to complete 
surveys on-site, the response rate was high (80%), and missing surveys likely reflected 
improper entry of unique identification numbers or accidental failure to complete one 
survey. 

Analysis 

Data from the pre-survey and post-survey were merged using a unique identifier 
provided by participants in each survey. Participant data that did not include both 
surveys were considered incomplete and were excluded from analysis. If multiple 
responses existed for participants, timestamps were used to select the earliest 
submission for the pre-survey (to avoid post-surveys mistakenly taken as pre-surveys) 
and the latest submission for the post-survey (to avoid pre-surveys mistakenly taken as 
post-surveys). All other submissions were discarded. Several tests were run on the data 
to answer pertinent research questions, and an explanation of each research question 
and its accompanying test(s) is now explained. 

RQi: False confidence and misconceptions. 

H 0 : There was no difference between self-evaluations of prior knowledge collected 
before the institute and after the institute. 

Hi: Self-assessments of prior knowledge collected before the institute were different 
than self-assessments of prior knowledge collected after the institute. 

In the pre-survey, participants were asked “How well do you understand each of the 
following concepts or movements?” and then were expected to self-evaluate their 
understanding of six open or general education knowledge domains (“Common Core”, 
“open education,” “copyright,” “fair use,” “copyleft,” and “public domain”) according to 
a 5-point Likert scale. It was believed that participants might initially rate themselves 
one way on these knowledge areas but that upon completion of the institute, they might 
come to realize that their initial self-assessments were incorrect. For this reason, the 
post-survey included the same question, which was reworded as follows: “How well did 
you understand each of the following concepts or movements before the institute ?” 
These data were analyzed using paired samples T-tests on each knowledge domain to 
determine if there was a significant difference between pre-survey assessments of prior 
knowledge and post-survey assessments of prior knowledge with the expectation that a 
negative change would reflect a realization on the part of participants that their initial 
self-assessments had been overstated or based upon a misconception of what the 
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knowledge domain entailed. When completing the post-survey, participants were not 
given access to their pre-survey assessments, which required them to self-evaluate 
without reference to their former assessments. In this analysis, the phrase “false 
confidence and misconceptions” is used to inclusively address all possibilities wherein a 
participant’s pre-survey assessment of prior knowledge does not match her post-survey 
assessment of prior knowledge and would include instances where participants might 
have forgotten the complexity of a topic. 

RQ2: Knowledge growth. 

H () : Participants reported no knowledge growth as a result of the institute. 

Hii Participants reported knowledge growth as a result of the institute. 

In the post-survey, participants were also asked to self-assess their final knowledge with 
the question “How well do you understand each of the following concepts or movements 
now?” in connection with the six open education knowledge domains mentioned above 
and were provided with the same 5-point Likert scale. Two sets of paired samples T- 
tests were run: one comparing pre-survey prior knowledge with post-survey final 
knowledge and the other comparing post-survey prior knowledge with post-survey final 
knowledge. It was anticipated that if knowledge growth occurred, both of these sets of 
tests would reveal significant differences. 

RQ3: Expectations and takeaways. 

H () : Valued takeaways from the institute matched initial expectations. 

Hi: Valued takeaways from the institute did not match initial expectations. 

In the pre-survey, participants were asked “What do you hope to gain from this institute 
(please rank with the most valuable at the top)?” and were provided with the following 
four items: 

1. Open content creation literacy (e.g., how to create open content) 

2. Relationships with other educators (e.g., building a professional 
learning community) 

3. Technology integration strategies (e.g., how to integrate technology x) 

4. Technical skills (e.g., how to use technology x ) 

All of these were topics addressed in the institute. In the post-survey, participants were 
again asked to rank these same four items in accordance with this question: “What was 
the most valuable knowledge or skills that you gained from this institute (please rank 
from most valuable to least)?” Paired samples T-tests were then run on each item with 
the expectation that a change in average ranking of an item would reflect a difference 
between participants’ initial expectations of the institute and actual takeaways. 
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RQ4: Time teaching. 

H C) : Time teaching has no effect on expectation, knowledge, or evaluation metrics. 

Ho Time teaching has an effect on expectation, knowledge, or evaluation metrics. 

In the pre-survey, participants were asked “How long have you been teaching?” and 
were provided with the following three options: “1 year or less,” “2-5 years,” or “more 
than 5 years.” A one-way ANOVA with Bonferroni post hoc test was then run with time 
teaching as the factor and each expectation, knowledge, and evaluation item from the 
pre-survey and post-survey as a dependent variable. It was expected that this test would 
reveal any cases where time teaching had an effect on survey outcomes. 

RQ5: Influences on overall evaluation. 

H 0 : There is no linear correlation between participants’ overall evaluations and specific 
evaluation items. 

H1: There is a linear correlation between participants’ overall evaluations and specific 
evaluation items. 

In the post-survey, participants were asked “How would you rate this institute?” and 
were then expected to evaluate the institute overall and in ten specific evaluation items 
according to a 5-point Likert scale. Categories included: instructor, support staff, 
schedule / organization, learning activities, your PLC, tech explorations, website / 
online resources, lab / venue, food / refreshments, and lodging. A stepwise linear 
regression model was then used with overall evaluation as the dependent variable and 
all ten specific evaluation items as the independent variables to determine whether 
linear correlations existed between specific evaluation items and the overall score, 
thereby revealing which specific evaluation items informed the overall rating. 


Findings 


Descriptive statistics revealed that participants believed their institutes to be highly 
valuable and effective. The average participant overall rating for the institute was 4.86 
on a 5-point Likert scale, and 44% of participants believed their institute was the best 
professional development experience they had ever experienced, and another 44% 
believed that it was much better than most other professional development experiences 
that they had experienced in the past. In their evaluations, participants rated all aspects 
of the institute highly, and participants strongly agreed that the institutes were a good 
use of their time, that they were of practical value to their classroom practice, and that 
the institutes encouraged them to think critically about technology integration (cf. Table 
1). Findings emerging from statistical analysis related to each research question now 
follow. 
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Table 1 

Descriptive Statistics of General Evaluation Items 



n 

Mean 

SD 

Overall rating 

80 

4.86 

•35 

Practical value 

79 

4-73 

•44 

Good use of time 

78 

4-77 

.42 

Encouraged critical thinking 

80 

4-59 

.61 

Comparative value a 

76 

4-49 

•57 


A This item was formulated on a 7-point Likert scale (M = 6.29, SD = .8), but results 
were converted to a 5-point scale to allow for uniformity in reporting. 


RQi: False Confidence and Misconceptions 

The comparison of pre-survey prior knowledge with post-survey prior knowledge 
yielded a number of significant differences between how participants initially evaluated 
their knowledge on topics related to open education and how they later came to assess 
their prior knowledge. In the cases of open education, copyright, fair use, and public 
domain, participants’ self-assessments went down in the post-survey, so we must reject 
the null hypothesis and conclude that self-assessments differed significantly before and 
after the institute for these cases (cf. Table 2). This finding suggests that initial 
participant self-assessments might have been based on false confidence or 
misconceptions about what the terms meant, but that as participants became more 
familiar with terms through the institutes, they came to recognize how little they 
actually knew before entering the institute. Differences on Common Core and copyleft 
were not significant, suggesting that the institute did not change participant 
understanding of what these terms meant (as is likely the case with Common Core) or 
that participants had no prior knowledge of the term (as is likely the case with copyleft). 

Table 2 

Comparison of Pre-Survey and Post-Survey Prior Knowledge Ratings 



Prior knowledge 
Pre-survey 

Post-survey 

Mean difference 

T 

Df 

Copyright 

2.76 

2.31 

-•45 *** 

-5-07 

79 

Fair use 

2.24 

1.8 

-.44 *** 

-5-39 

79 

Public domain 

2-45 

2.03 

- 43 *** 

-4.11 

79 

Open education 

2.25 

1.91 

- 34 *** 

-4-36 

79 

Common Core 

3-8 

3-83 

•03 

•41 

79 

Copyleft 

1.38 

i-34 

-.04 

-.56 

79 


*** Denotes significance at the p < .001 level. 
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RQ2: Knowledge Growth 





The comparison of pre-survey prior knowledge with post-survey final knowledge and 
also the comparison of post-survey prior knowledge with post-survey final knowledge 

yielded significance in every case (cf. Table 3 and Table 4). 

Thus, we must reject the null 

hypothesis and conclude that participants reported knowledge growth as a result of the 

institute in every domain. 





Table 3 





Comparison of Pre-Survey Prior Knowledge and Post-Survey Final Knowledge 


Pre-survey prior 

Post-survey final 

Mean 

T 

Df 

knowledge 

knowledge 

difference 



Copyleft 1.38 

3-89 

2.5 *** 

26.1 

79 

Open education 2.25 

4.18 

1 93 *** 

19.2 

79 

Public domain 2.45 

4.26 

1.81 *** 

13-85 

79 

Fair use 2.24 

3-94 

1 rj *** 

17-93 

79 

Copyright 2.76 

4-03 

1.26 *** 

12.22 

79 

Common Core 3.8 

4-1 

2 *** 

4.16 

79 

*** Denotes significance at the p < .001 level. 




Table 4 





Comparison of Post-Survey Prior Knowledge and Post-Survey Final Knowledge 


Post-survey 

Post-survey final 

Mean 

T 

Df 

prior knowledge 

knowledge 

difference 



Copyleft 1.34 

3-88 

2.54 *** 

27.48 

79 

Open education 1.91 

4.18 

2.26 *** 

24-59 

79 

Public domain 2.03 

4.26 

2.24 *** 

18.7 

79 

Fair use 1.8 

3-94 

2.14 *** 

22.03 

79 

Copyright 2.31 

4-03 

1 71 *** 

17.24 

79 

Common Core 3.83 

4-1 

2g *** 

4.67 

79 


*** Denotes significance at the p < .001 level. 


RQ3: Expectations and Takeaways 

The comparison of pre-survey expectations with post-survey outcomes yielded 
significant results in every case (cf. Table 5). Thus, we must reject the null hypothesis 
and conclude that valued takeaways did not match initial participant expectations. 
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Table 5 

Comparison of Pre-Survey Expectations and Post-Survey Outcomes 



Pre-survey 

expectations 

Post-survey 

outcomes 

Mean 

difference 

T 

Df 

Pic 

3-3 

2-45 

- 86 *** 

-5.82 

5 

Open education 

2-39 

2 

-■39* 

-2-3 

55 

Technology skills 

2.71 

3-04 

■32* 

2.07 

55 

Technology integration 

1-59 

2.5 

*** 

6.1 

55 


Note: Since these are ranked items, a lower number indicates a higher score (1 meaning 
first, 2 meaning second, etc.). 

* Denotes significance at the p < .05 level. 

*** Denotes significance at the p < .001 level. 


To clarify this finding further, if we were to list expectations and outcomes in 
accordance with their rankings, we would see that the largest changes occurred in the 
cases of technology integration, wherein participants expected to learn about 
technology integration but did not count it as a valuable outcome, and PLCs, wherein 
participants did not expect their PLCs to be valuable but then evaluated them highly as 
an outcome (cf. Table 6). 

Table 6 

Expectations and Outcomes in Ranked Order 


Expectations from pre-survey 

Outcomes from post-survey 

1 

Technology integration 

1 

Open education 

2 

Open education 

2 

Professional learning community 

3 

Technology skills 

3 

Technology integration 

4 

Professional learning community 

4 

Technology skills 


RQ4: Time Teaching 

ANOVA tests on knowledge items generally did not reveal differences between 
participants when grouped according to time teaching or teacher veterancy. The only 
significant main effects between groups were found on the Common Core and fair use 
items in the pre-survey and on the Common Core item in the post-survey (cf. Table 7). 
Bonferonni post hoc tests revealed that this difference can be attributed to the least 
experienced teaching group, which self-assessed lower than more experienced groups in 
all three metrics, with an average difference ranging between .71 and 1.14 points on the 
5-point scale (cf. Table 8). 
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Table 7 

Main Effect of Time Teaching on Knowledge Items 



SS 

df 

Mean Square 

F 

Pre-survey Common Core 

6.96 

2 

3-48 

5.89* 

Pre-survey fair use 

6.12 

2 

3.06 

3.71* 

Post-survey Common Core 

3-68 

2 

1.84 

3.22 * 


* Denotes significance at the p < .05 level. 


Table 8 

Main Effect Comparison of Less and More Experienced Teachers on Knowledge Items 



Comparison of 1 Year or Less Group to 


2-5 Years Group 

5+ Year Group 

Mean 

difference 

Standard 

error 

Mean 

difference 

Standard 

error 

Pre-survey Common Core 

+1.14 * 

•36 

+1 * 

•3i 

Pre-survey fair use 

+1.14 * 

.42 

+.8 

•36 

Post-survey Common Core 

+.71 

•35 

+.77* 

•3 


* Denotes significance at the p < .05 level. 


RQ5: Influences on Overall Evaluation 

Participants rated sessions highly across all ten specific evaluation items, but the 
stepwise linear regression revealed that three specific evaluation items (activities, 
instructor, and website) significantly predicted overall ratings (cf. Table 9). The 
regression model for all three of these predictors also explained a significant proportion 
of variance in overall ratings, R 2 = .649, F( 3, 68) = 41.94, p < .001. Of these factors, 
activities and instructor had a positive linear correlation with overall ratings, while 
website had a negative linear correlation. All other factors were excluded from the 
regression model due to lack of significance. 
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Table 9 

Regression Model of Activities, Instructor, and Website Prediction upon Overall 
Evaluation 


Unstandardized coefficients 


Model B Std. Error Beta T 


(Constant) 

i-43 

•4 


3-54 *** 

Activities 

•43 

.06 

•65 

7.08 *** 

Instructor 

•54 

.09 

•49 

6.03 *** 

Website 

-.27 

•07 

-•34 

38 *** 


*** Denotes significance at the p < .001 level. 


Implications 


A variety of implications arise from these findings. First, it may be concluded that the 
institutes were considered to be a valuable learning experience for participants and that 
utilizing this type of approach for developing open education literacies in practicing 
teachers can yield positive results and help to address this need (cf. Tonks, Weston, 
Wiley, & Barbour, 2013). This finding was corroborated in the knowledge growth 
analysis, which found that participants’ self-evaluations on specific knowledge items 
increased significantly both when comparing pre-survey prior knowledge with post¬ 
survey final knowledge and when comparing post-survey prior knowledge with post¬ 
survey final knowledge. Making both of these comparisons allowed us to determine 
more surely that participants’ knowledge grew than would have been possible by simply 
asking participants to reflect on their learning. 

Second, it seems that part of the challenge with open education revolves around 
misconceptions and false confidence related to key components. It is telling that 
participants changed their initial ratings of themselves on knowledge of copyright, fair 
use, public domain, and open education between the pre-survey and the post-survey 
and rated themselves lower on prior knowledge after having experienced the institute. 
This corroborates our anecdotal findings that teachers tend to believe that they 
understand what these concepts mean and what they entail, but that upon examination 
and the completion of focused learning activities, participants come to recognize that 
they did not understand the concepts very well to begin with. This is problematic for 
open education, because it is difficult to appeal to a need when teachers do not 
recognize that a need exists. If teachers already believe that they understand copyright 
and fair use, for instance, then they have no impetus to learn about these concepts and 
may consider themselves to be open educators when in fact they have very little 
understanding of what this entails and what it means to share in open ways utilizing 
copyleft or Creative Commons licensing. Further research in this area would be valuable 
for gaining a more nuanced understanding of misconceptions and false confidence via 
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qualitative analysis, but such analyses were beyond the scope of the current study and 
were not essential for answering the research questions. 

Third, we found it noteworthy that perceived importance of both open education and 
professional learning communities increased through the course of the institutes, while 
importance of technology integration and skills decreased. This suggests that if we truly 
seek to create open participatory learning ecosystems (Brown & Adler, 2008, p. 31), 
teachers need experiences like these institutes that allow them to experience 
collaborations with other teachers in an open manner. It seems dubious that system- 
wide collaborations (Carey & Haney, 2007) can occur otherwise, because open 
education requires teachers to rethink fundamental aspects of how they operate as 
educators, to reevaluate basic collaborative practices, and to share in ways that may be 
new and uncomfortable. In addition, as teachers recognize collaborative potentials with 
one another across traditional school and district boundaries and recognize that they 
have value to contribute to the profession through sharing, this may help to counteract 
deskilling influences upon teachers, wherein they are relegated to serving as technicians 
rather than professionals (Gur & Wiley, 2007). 

Fourth, though there is no theoretical basis for assuming that innovation adoption is 
correlated with age factors (cf. Rogers, 2003), it has been our experience that many 
advocates for innovation and technology integration resort to a narrative of innovation 
which considers younger teachers to be more willing to innovate than their more 
experienced peers. Our findings, however, reveal that time teaching had no impact on 
participants’ expectations of the institutes or their evaluations of the experience, which 
means that veteran teachers responded just as positively to the learning activities as did 
their less experienced counterparts. The only significant differences we found related to 
two knowledge items: Common Core (pre-survey and post-survey) and fair use (pre¬ 
survey only). In the case of Common Core, it makes sense that more veteran teachers 
would self-assess higher than less experienced teachers, because they have had more 
experience teaching and adapting to new standards or ways of teaching and also work in 
districts that have devoted a sizable amount of training to Common Core, while the less 
experienced teachers would have just recently completed their teacher education 
programs and likely would not have completed many district or school level trainings. 

The difference with fair use, on the other hand, reveals that veteran teachers entered the 
institute with greater perceived knowledge of fair use than did their novice counterparts 
but that this difference disappeared by the end of the institute. This means that either 
veteran teachers truly began the institute with a greater knowledge of fair use than their 
novice counterparts or they had more false confidence in this regard. Given the fact that 
training on issues of copyright and fair use are uncommon for teachers, we believe that 
the latter interpretation is likely more accurate and that as teachers spend time in the 
classroom and use copyrighted works, they develop a false sense of confidence related to 
fair use. This interpretation is corroborated by the fact that when novice teachers and 
veteran teachers self-assessed their prior knowledge on the post-survey, differences 
between groups disappeared, meaning that after participants had focused training 
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related to fair use, they self-evaluated themselves equally low on initial knowledge. This 
is problematic, because it suggests that as teachers gain experience in the classroom, 
they also develop a false sense of confidence related to fair use and therefore likely begin 
utilizing copyrighted materials in ways that may not be permissible. This also means 
that although the development of open education literacies is essential for ongoing 
diffusion (Tonks, Weston, Wiley, & Barbour, 2013), teachers may not recognize the need 
to learn more about open education, because they assume that they already sufficiently 
understand these topics. 

And fifth, if open education leaders seek to help K-12 teachers develop literacies 
necessary to utilize open educational resources in their classrooms and to share their 
own creations through open practices, then we need to understand what factors 
influence these teachers’ ratings of learning experiences toward this end. From our 
results, we find that the learning activities themselves, which involved collaborative 
group work with other professional educators, and the instructor, who modeled open 
educational practices and facilitated collaborative learning, were the most important 
factors for creating a positive open education experience. 

Interestingly, though participants provided anecdotal feedback that the website and 
online resources were valuable, their ratings in this regard are negatively correlated with 
overall satisfaction with the institutes. The reason for this is unknown, but it may be 
that those teachers who valued the ability to peruse resources on their own and to learn 
at their own pace via provided online resources found the face-to-face institute to be less 
valuable, whereas those who found the online resources to be less useful needed to rely 
more heavily on the institute and valued the experience more as a result. This may mean 
that some educators might be more effectively introduced to open education via online, 
asynchronous learning experiences, while others may be more effectively reached 
through face-to-face, synchronous experiences. 


Conclusion 


Given these findings and implications, we conclude that this type of open education 
institute can be valuable for practicing teachers if coupled with effective and 
collaborative learning activities and a strong instructor to model open education 
practices and collaborative learning for participants. We also conclude that there maybe 
a number of misconceptions related to open education that make it difficult for 
practicing teachers to recognize the need for this type of work but that as they 
participate in learning activities increasing their knowledge of concepts like copyright, 
fair use, public domain, and copyleft, teachers come to recognize the value of these 
subjects and to value learning experiences devoted to them. And finally, advocates of 
open educational practices should eschew narratives of innovative change that 
categorize educators based upon veterancy factors and recognize that all teachers want 
to innovate and share; teachers merely need learning experiences that empower them to 
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overcome false confidence and misconceptions in a manner that is positive and that 
treats them as competent professionals. 
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